1. Field of the Invention
The present invention relates to an image processing apparatus, an image capturing apparatus, an image processing method, and a program, and more specifically, to an image processing apparatus, an image capturing apparatus, an image processing method, and a program capable of generating images to display 3-dimensional images (3D images) using the plurality of images captured while a camera is moved.
2. Description of the Related Art
In order to generate a 3-dimensional image (also called a 3D image or a stereo image), images have to be photographed at different observing points, that is, left-eye images and right-eye images have to be photographed. A method of photographing the images at the different observing points is broadly classified into two.
A first method is a method of using a so-called multi-lens camera capturing a subject simultaneously at different observing points using a plurality of camera units.
A second method is a method of using a so-called single lens camera capturing images continuously at different observing points using a single camera unit while the image capturing apparatus is moved.
For example, a multi-lens camera system used in the first method has a configuration in which lenses are disposed at spaced positions to photograph a subject simultaneously at the different observing points. However, the multi-lens camera system has a problem in that a camera system is expensive since the plurality of camera units is necessary.
On the contrary, a single lens camera system used in the second method includes one camera unit as in a camera according to the related art. A plurality of images is photographed continuously at different observing points while a camera including one camera unit is moved and the plurality of photographed images is used to generate a 3-dimensional image.
Accordingly, when the single lens camera system is used, the system with one camera unit can be realized at a relatively low cost, as in a camera according to the related art.
In “Acquisition of Distance Information Using Omnidirectional Vision” (Journal of the Institute of Electronics, Information and Communication Engineers, D-II, Vol. J74-D-II, No. 4, 1991), a technique according to the related art describes a method of acquiring distance information on a subject from an image photographed while a single lens camera is moved. In “Omni-Directional Stereo” (IEEE Transaction On Pattern Analysis And Machine Intelligence, VOL. 14, No. 2, February 1992), the same contents as those of “Acquisition of Distance Information Using Omnidirectional Vision” (Journal of the Institute of Electronics, Information and Communication Engineers, D-II, Vol. J74-D-II, No. 4, 1991) are also described.
“Acquisition of Distance Information Using Omnidirectional Vision” (Journal of the Institute of Electronics, Information and Communication Engineers, D-II, Vol. J74-D-II, No. 4, 1991) and “Omni-Directional Stereo” (IEEE Transaction On Pattern Analysis And Machine Intelligence, VOL. 14, No. 2, February 1992) describe the method of acquiring the distance information of a subject using two images obtained through two vertical slits by fixing a camera on the circumference placed at a given distance from the rotation center of a rotation table and photographing images continuously while rotating the rotation table.
As in “Acquisition of Distance Information Using Omnidirectional Vision” (Journal of the Institute of Electronics, Information and Communication Engineers, D-II, Vol. J74-D-II, No. 4, 1991) and “Omni-Directional Stereo” (IEEE Transaction On Pattern Analysis And Machine Intelligence, VOL. 14, No. 2, February 1992), Japanese Unexamined Patent Application Publication No. 11-164326 discloses a configuration in which a left-eye panorama image and a right-eye panorama image applied to display a 3-dimensional image are acquired using two images obtained through two slits by installing a camera placed at a given distance from the rotation center of a rotation table and photographing images while the camera is rotated.
In the plurality of techniques according to the related art discloses the method of acquiring the left-eye image and the right-eye image applied to display a 3-dimensional image using the images obtained through the slits when rotating the camera.
On the other hand, there is known a method of generating a panorama image, that is, a 2-dimensional image with a horizontally long shape by photographing images while moving a camera and connecting the plurality of photographed images. For example, Japanese Patent No. 3928222 and Japanese Patent No. 4293053 disclose a method of generating a panorama image.
When the 2-dimensional panorama image is generated, the plurality of images photographed while the camera is moved is used.
“Acquisition of Distance Information Using Omnidirectional Vision” (Journal of the Institute of Electronics, Information and Communication Engineers, D-II, Vol. J74-D-II, No. 4, 1991), “Omni-Directional Stereo” (IEEE Transaction On Pattern Analysis And Machine Intelligence, VOL. 14, No. 2, February 1992), and Japanese Unexamined Patent Application Publication No. 11-164326 describe a principle of acquiring a left-eye image and a right-eye image as 3-dimensional images by applying the plurality of images photographed in a photographing process, like the process of generating the panorama image, and cutting and connecting images at predetermined areas.
However, these documents describe the principle at most. For example, when a user holds a camera with the hands and photographs a panorama image while swinging the camera, a condition or the like of reliably acquiring the left-eye image and the right-eye image applicable to a 3-dimensional image is not described. When the user holds a camera with the hands and photographs a plurality of images while moving the camera, a movement speed or the like of the camera is varied depending on the situation. A necessary condition or like of acquiring the left-eye image and the right-eye image applicable to the 3-dimensional image under various photographing conditions is not clearly described, for example, in the above documents according to the related art.